The present invention relates to automated security analysis of hardware and/or software.
Systems with network capability and software running on such systems are vulnerable to various network attacks. If a system is expected to provide services that meet a certain quality standard, it should continue to provide services that meet this standard despite network-based attacks. For example, many hosted network service providers commit to certain levels of uptime or availability for their services, and they would like to meet these levels even if under malicious network attacks. As another example, critical systems such as medical devices must respond to emergencies promptly, even under malicious network attacks. Therefore, it is important to test the security of a system and characterize its robustness against malicious attacks.
One way to test system security is to put the system under attack and analyze the behaviors of the system under attack to discover vulnerabilities. This process, which is known as “security analysis,” can be performed using various methodologies. One methodology is to treat the device-under-analysis (DUA) as a black box. Under this methodology, the DUA is analyzed via the interfaces that it presents to the outside world (for example, by sending messages to the DUA). As a result, it is not necessary to access the source code or object code comprising the DUA.
Attacks can cause various failures in the DUA. Some of the failures are fatal (also known as hard failures) and disable the DUA. Some other failures are nonfatal (also known as soft failures). Soft failures do not immediately disable the DUA but may cause high CPU utilization, memory leaks, an increase in response times and/or temporary unresponsiveness. Soft failures often only affect the DUA internally and do not cause material behavioral changes of the DUA. Therefore, soft failures are hard to identify.
Nonetheless, soft failures can have substantial impacts on the robustness of the DUA. For example, they can deteriorate the system performance of a DUA slowly and eventually crash the DUA. Therefore, in order to measure the robustness of a DUA against malicious attacks, it is desirable to characterize (or quantify) the soft failures in the DUA caused by the attacks. Because the soft failures are mostly manifested internally and hard to identify, they are also hard to characterize.
Thus, there is a need for a system and method to identify and characterize soft failures of a DUA under attack.